f>1 

+ 

T 
+ 



II 




d 



TrainEnsemble (A) 



Sub-element A.l 



20 



N,E,B,X,s<-l 
32 




TrainStage (B) 



s ^ ' opt 




v N,sMT,olderr 




IF finished = TR JE 
. v ■ 
-4 



IF finished = FAL SE 
36 ~ 




I >''■■•' V I 



FIG. 6 



32 

Train Stage (B) 



n,e,s,bX 



B.l Copy training sets for this stage into individual sets 

{t;,... 4 t,;}^-t; 

B.2 Compute generalisation error estimates for each training vector 

For every training vector (n=l TO N) in the original training set repeat 
For every epoch (e=l TO E) repeat 



B.3 Aggregate the ensemble generalisation error estimates 

For every epoch (e-1 TO E) repeat 
Compute: 



B.4 Find the best value for e for each network in the ensemble 



Compute: 



e n 




e o P t <-argmin(A e ) 




1 s 

Copt 



.7 



s 

opt 



PropStage (C) 



C.l Compute ensemble outputs for each training example for this stage 

For every training vector {n=l TO N) in the original training set 
Compute: 

\B 



M" ~<r- 
n 



T B r" 



M 
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N,s,M 9 T,oIderr 



CombineStages (P) 




D.l Set new variable as upper bound on number of stages so far 

num stages <— s 



For ever)' training vector (n=J TO TV) in the original -training set 
Compute: 



D3 Calculate staged ensemble generalisation error 



D.4 If no improvement finish training 

lFs=l 

olderr <— newerror 
ELSE IF {newerror > (d* olderr)) 

finished 1 
ELSE IF {newerror < olderr) 

olderr <— newerror 



D.2 Sum ensemble outputs across stages 




n 



newerror i^—YLS*' -S ») 2 




fig. 9 I finished 
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AdaptSet (E) 



N, s, M, T, 



E.l Set new variable as upper bound on number of stages so far 

numstages <- s 

E.2 Sum ensemble outputs across stages 
For every training vector (n=l TO N) in the original training set 
Compute: 



E.3Adapt training set 

For every training vector (77= i TO N) in the original training set 
Compute: 



FIG. 10 



